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ABSTE ACT 

'T'he service receiving first priority in the new 
regional medical library (emL) nrogram is that of document delivery. 
Since this is a federallv sponsored proaram^ ^onJ •'•or i nq devices 
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f the program is accomplishing the 
1 information. Tn this paper, two 
perspectives are used on which to base a design for measurement of 
document delivery services and on which to base performance 
standards: (1) the cost to the user of this service, and (2) the 

efficiency of the procedures. One overriding factor governs the 
design for measurement: a more precise definition of what variables 
are to be included with different measurements is necessary before 
any realistic performance standards can be established. Three sources 
of data are suggested: (1) the records generated in making requests, 

(2) the user time involved, and (3) the operational procedures in 
providing the service. Eour priorities for analyzing this data are 
then suggested: (1) determining who uses the service, (2) analyzing 

reasons for unfilled requests, (3) collecting and analyzing data on 
reauest and loan processing time, and (4) establishing the actual 
user real time cost in using the service for different 
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If no quantitative standard is defined against which to make 
comparisons, measurements must be stated in terms of social values; that 
is, an object or event is good or bad according to the taste or authority 

of the measurer. Social values are considered by many to be incommensu- 
rabl e . 



Theoretically this may be true; but in real life 
i ncommensurabi es are commensurable. Only a criterion 
of judgment and a system of weighting are needed. 

In nature the criterion is survival.... Natural 
selection commensurates the incommensurables . The 
compromise achieved depends on a natural weighting of 
values of the variables. (1) 

Bureaucratic institutions are organized to carry out tasks for 
society which could not be accomplished by Individual effort. (2) In 
Western society the survival of Institutions, the criteria of judgment on 
the "value'* of institutions,, is dependent upon the market place and the 
ballot box. The analogy between nature and the market place has Its 
limitations, as do all analogies. Bureaucratic organizations are corrupt- 
ible and self-protective; the products and services of an Institution 
supported by society are often judged In terms of the vested interest of 
the bureaucratic organization rather than its use to society. 

The growing interdependence of our civilization requires an 
increasing Institutionalization of social functions. The bureaucratic 
organizations to supply the needed products and services are generally 
created in one of two ways. The organization evolves through the re- 
arranging and branching of an existing bureaucracy. The alternative method 
usually arises because of the recognition that existing organizations and 
agencies are Inadequate and incapable of evolving to meet needs. An 
administrative or legislative action Is taken to set up a J'new" organi- 
zation. Whether one or the other of the two general approaches is used, 
decisions as to priorities have to be established. Those given the 
responsibility to make such decisions have to place values on (l) the 
usefulness of the product and service, (II) the resources available to 
produce the product or service, and (III) the alternative methods of 
utilizing the available resources. The more precise the information 
available on (i) and (li) to those who must make organizational decisions, 
the fewer are the alternatives In organizational procedures and methods. 



(1) Harlan, G. The Tragedy of the Commons. Science 162:1243-8. 
13 December 1968. 

(2) Blau, P.M. Bureaucracy in a Modern Society. New York, Handom 
House, 1956. 
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This paper is a discussion of some methods and approaches that 
can be used to acquire information on one service, document delivery, to 
be given through the creati(jn of a new library organ i iiiat ion , the regional 
medical library (RML) . Specifically, it is an attempt to determine (i) 
what aspects of a RML document delivery service can be measured quantita- 
tively, and (ii) what possible significance the measurements may have in 
evaluating the service. 

CONDITIONS AND ASSUMPTIONS 



Med i ca 1 Library Ass i stance Act of 1965 



The new institution, RML, was described by Congress only in the 
most general terms. Although it might be interesting to know what the 
intentions of Congress and the writers of the Act were, interpretations have 
to be made from the language of the Act itself. 



1) Apparently, it was assumed that the existing health science 
library complex was not providing adequate service to 
research workers and professional people, hence the need 
for a new kind of institution. 



2) That RML is a new kind of organization from a national 
viewpoint is clear in that one of the conditions for 
federal support is that libraries "modify and increase 
their library resources ... to provide supportive services 
to other libraries as well as individual users of library 
serv ice". 



3) The program to create these new institutions is through 
a granting mechanism: this entails a procedure of review 
and evaluation that is quite different from the situation 
where Congress gives the authority to government agencies 
to issue contracts . 

4) I f no RML can be created for a geographic area through the 
granting mechanism, the National Library of Medicine is 
"authorized to establish, as a branch of the National Li- 
brary of Medicine, a regional medical library to serve the 
needs of such an area". 

5) Only one service is mentioned in the Act, that of document 
delivery; it is to be given without cost to the user, and 
facilities must be available to make facsimile copies. 

The intention of the Act appears to be that a new kind of docu- 
ment delivery service be provided to libraries and individual users through 
the reorganization of existing libraries. Two things need to be measured: 
(i) the effectiveness of the institutional reorganization to accomplish the 
purpose of the Act, and (ii) the adequacy of the document delivery service 
to the user . 
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Document del i very serv i ce from the user v i ewpo i nt 

A document delivery service is ultimately an individual service. 
Any measurement of the service must include its efficiency in terms of 
the user's time. Obviously, some kind of criteria must be established 
on users' qualifications and needs. Some criteria can be established 
ja priori and others can only be determined after information is available 
to make j udgments . 

1) Who is a qualified user? Certainly the Medical Library 
Assistance Act has not socialized all medical libraries 

to make materials accessible without cost to a 1 1 individuals 
who claim a need. Eventually, when the entire scholarly 
record is in machine readable form with terminals located 
in every office and every hospital ward, instant access may 
be possible. Until that time a priority method must be 
created. 

2) Are there limitations on what may be supplied? A regional 
document delivery service should not replace local library 
service, nor should a RML take over the function of a 
publisher's distribution system. 

3) Should time priorities be established in supplying docu- 
ments? Can users be grouped in some way according to 
their needs? Should all requests be answered within a 
set time limit, or should delivery be measured against a 
proportionate standard; i.e., a minimum percentage answered 
within one day, etc . 

In summary, a document delivery service on a regional basis has 
to be judged in terms of user cost 

...if delays in access increase user cost and the 
aim is to minimize this cost, then what must be 
minimized Is the average access time to all publi- 
cations and patron uses--those locally owned as well 
as those borrowed or photocopied. (3) 

Cond i t i ons of measurement 

Although the above questions can be asked, answers are not avail 
able. The determination of user cost in gaining access to documents has 
never been systematically studied. The sociological factors involved in 
this aspect of scholarly and educational communication are complex. The 
only effective measure so far devised to test user cost is through the 
administrative mechanism of settling upon a dependable service. If the 
demand for the service increases, then apparently the users find it a more 
suitable way to secure documents than alternative methods. 




(3) Williams, G. Library cost models: owning versus borrowing serial publi- 
cations. Office of Science Information Service, November 1968. 
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If the market place is used as the measuring device for user 
cost or satisfaction then the deciding factor about the service is 
whether It "pays for Itself". RMLs are not commercial organizations 
and hence a demonstration of monetary profit Is unsuitable as a measure. 
Every publically supported library has constraints In establishing any 
service; (i) the available funds that can be diverted to support the 
service, (li) the number and quality of staff, (III) the extent of Its 
resources, and (Iv) the availability of space. Any service that is 
organized which Ignores any one of the above constraints Is likely to 
be undependable. The market place In this situation cannot be used as 
a rel lable measure . 

The measurement of RML document delivery service has to be 
viewed as a bureaucratic function. (I) Does It maximize efficiency 
of administration In terms of the external constraints? (II) As a 
method of organizing social conduct for administrative efficiency. Is 
It accomplishing its stated objective? To reiterate: A document delivery 

service must be designed to function efficiently within the constraints 
of the Institution; and efficiency Is measured In relation to the maxi- 
mization of these constraints, and the effectiveness of the administrative 
efficiency Is evaluated in terms of Its Increased use. 

Assuming that a document delivery service Is organized efficiently; 
that Is, the facilities are fully exploited, an increased use places demands 
on the facilities which can produce three results, (i) reduce the amount 
of service by limiting the clientele qualified to receive the service, 

(II) alter one or more of the constraints which In turn requires a revision 
in administrative organization, or (ill) reduce the efficiency and hence 
dependability which will In turn reduce the demand. On the other hand, 
once a document delivery service Is established and operating with admini- 
strative efficiency, any attempt to expand It quantitatively, again, 
requires an alteration of one or all of the constraints of funds, staff, 
resources, and space. If the expansion of service Is to be accomplished 
with administrative efficiency, then Information on which (and how much) 
of the constraints to alter must be available to make rational plans. 

Qua 1 1 tv of measurements 

A document delivery service Is an ongoing operation and as such 
is a dynamic system. The act of studying should not alter the object of 
study. Further, the results of the study should allow for assessing 
changes In the system over a period of time, and above all, for a national 
service as set up by RMLs, a comparison among different environments Is 
needed. Other requirements are needed in undertaking to measure and to 
evaluate an ongoing system where the objective is to Improve and/or expand 
the system. (4) 



(4) For a fuller discussion on methodologic approaches, see Orr, R. H., 
and others. Development of Methodologic Tools for Planning and 
Managing Library Services. Bulletin of the Medical Library Associ- 
ation 56:235-240, 1968. 
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1) The data obtained by any method should be reliable, 
that is, reproducible if the data is to be used for 
planning and management decisions. 

2) The methods of data collecting should have "face" 
validity to all those concerned with the service, 
including the user as well as the administrator. 

3) The methods should be practical, in that 

a) the cost of collecting the data are reasonable, 

b) their execution should not interrupt a library's 
normal operation, and 

c) the user for whom the data is ultimately to be 
evaluated should not be burdened, 

4) The measurements should be applicable to libraries, regard- 
less of differences in size, location, environment, or 
details of internal operation. 

5) The methods of data collecting at least should be suit- 
able for application by a library's own staff, rather 
than being useful only in the hands of outside "experts". 

WHAT IS MEASURABLE 

A modern myth of science is that given enough funds and personnel, 
any problem can be solved. Society, in its congressional wisdom, has not 
always given scientists the wherewithal to solve all problems. Undoubtedly, 
a massive study In which subject Interest profiles coupled with descriptions 
of all work activities of all health professions were related to the 
nation's health science Mbrary facilities and manipulated through some 
yet undeveloped technique of game theory or theory of groups could provide 
us with a temporary solution to the best way to deliver documents to all 
who need them. We live, however, in a statistical, irrational, institutional 
world. We must live within the conditions and requirements discussed above 
if any Immediate gain is to be obtained. If the perspective developed so 
far is accepted, there are but two general ways to obtain meaningful data 
about document delivery services, (i) on its use and (ii) on the organi- 
zational efficiency In providing the Information. A priority of the possi- 
ble elements for study is discussed below. 

Source of Data 

Any document supplied through RML will produce records. The first 
step In any measurement or monitoring procedure of document delivery would 
be to determine if data generated through normal operations can be used. 

The A.L.A. Interlibrary Loan Request form contains the minimum information 
that must be available to fill a request efficiently whether the request 
is from an institution or an Individual. 
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1) Source of request (If other than an individual) 

2) Requesting individual 

3) Status of individual 

4) Specialty of administrative unit with which requester 
is associated. 

5) Title of publication 

6) Date of publication 

7) Source of citation 

8) Lending library, or in the case of referrals, libraries 

9) Reason for non supply 

Very little organizational data on the library procedures for the 
.upply of documents can be secured from the request data. Such data will 
have to be secured through other methods. 

Use of document del i very service 

The total number of requests processed Is obviously an important 
'statistic" in determining use. This number, other than showing an increase 
)r decrease, has very little qualitative information which would provide a 
means for (i) assessing the relative importance in, say. Improving health 
v;are, or (ii) evaluating for planning and management decisions. Other 
information needs to be added. Each of the kinds of information generated 
with a request Is discussed for its particular relevance for measuring 
and monitoring the use of service, 

1) Source of request . An obvious question in any marketable 
service is, who will buy it. If, for example, it is found that 50% of 
the requests are generated by industrial and commercial agencies, the docu- 
ment delivery service supported by federal funds is in effect a subsidy 
to industry; or, if it is found that 50% of the requests are generated by 
hospitals, but that the number of hospitals making such requests constitute 
only 2% of the nation's hospitals, a whole series of questions need to be 
asked, e.g., should some service be planned for the remaining 98%? This 
Institutional information is probably now being tabulated in most libraries 
because of the need for billing. An analysis of this information should 
receive a high priority for the following reasons: 

a) As a public supported program, a justification is 
needed for continued support; who uses the service 
(institutionally) is important to administrators and 
Congressmen ; 

b) As a relatively simple means to evaluate the Impact of 
the document delivery program; 
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c) As a means to judge future directions for improvement; 
and 

d) Is any individual institution or group of institutions 
taking advantage of a publicly supported service to the 
detriment of others. (5) 

The actual analysis, once a consistent system is devised, is 
amenable to standard statistical procedures that are available from 
nearly all computing centers, Summaries can be prepared to include 
(i) types of institutions, (ii) number of requests by type, (ill) aver- 
ages, ratios, and percentages of requests and institutions. 

^ ^ 2) Requesting individual . Library service, In spite of what 

librarians may wish to achieve, is not used by everyone. Only a relative- 
ly small proportion of any group finds library service of use to them. 

A great deal of time could be spent in Investigating why such a situation 
exists. Although the data is available from requests, the cost of tabu- 
lating and subsequently analyzing it on any routine basis would appear to 
go beyond the methodologic limitations discussed above. 

3) Status and/or professional affiliation . Again, the tabulating 
of these data would appear to require an inordinate amount of time for the 
possible conclusions that could be reached. This might be done on a sampl- 
ing basis; however, to place such data in a perspective it must be secured 
from many institutions. The information would be useful for essentially 
the same reasons outlined under 1) above. Since RMLs are now only begin- 
ning, it would appear that other data should receive priority for analysis. 

1" * ^ Q ^nd date of pub 1 i cat i on requested . Many studies have 
been undertaken to determine the most heavily requested biomedical title:. 
Each of these studies was a major undertaking and certainly such use 
information could not be tabulated as a routine operation. Perhaps some 
sampling technique could be devised or some machine readable record 
generated with each request. In both instances a careful design would 
have to be made to result in useful Information which is different from 
that already available from other studies. 

5) Source of citation . One of the most neglected aspects of 
completing an interlibrary loan request Is verification. Although 
difficulties do not arise if (l) the citation is correct and (li) the 
lending library owns the item, a great deal of librarians' time at 
lending libraries is spent dealing with Incomplete citations for which 
there is no verification or source of rcifeirence. Because of the expense 



(5) For example, from two studies made in the last two years, It was 
suggested that any library that must borrow a title more than six 
times per year should consider the purchase of the title since the 
cost to borrow this many times equals the cost of owning the title, 
see Czlske, C. and Pings, V. A Study of Interlibrary Loans at Sinai 
Hospital of Detroit, Ju 1 y-December , 1965* Wayne State University 
Medical Library, Report No. 21. August 1966; see al so Wi 1 1 iams , op . c j t . 
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involved in processing incomplete requests, some endeavor should be made 
to determine if any pattern can be found in (i) where requesters obtain 
information about the document they want and (ii) what bibliographic 
tools are used by different institutions in verifying requests. Perhaps 
if these two elements were correlated with types of institutions making 
requests and dates of publications wanted, some standard performance 
level could be established which could be applied in determining whether 
requests should be rejected or whether assistance should be given to those 
submitting requests. Further, criteria might be set for the quality and 
extent of reference collections that should be maintained in different 
sizes and types of health science libraries. Agah*i, chis kind of study 
would involve a great deal of tabulating and analysis which could not be 
done on a continuing basis in most RMLs. 

6) Lend i ng library . If unfilled requests are to be referred 
from one resource library to another and between RMLs some record of the 
number of such referrals must be created, if for no other reason than for 
accounting purposes. Certainly, referring requests from one library to 
another is less expensive to society than the return of a request whicn 
in turn has to be sent to another library. Although this makes "admini- 
stiative sense", such a switching procedure has to be justified ecomonical- 
ly. 



7) Reasons for non-supply of requests . Inteflibrary loan studies 
rarely analyze the reasons requests are not filled. Users may be impressed 
on occasion with the amount of material they can receive through a document 
delivery service, but only a few negative experiences are sufficient for 
them to seek an alternate source of information. As a means for measuring 
or monitoring a document delivery service the analysis of unfilled requests 
is far more sensitive than counting and tabulating filled requests. If a 
lending library is supplying at least at the 70% level, quantitatively the 
analysis of unfilled requests can reveal a great deal of management and 
performance information. 

a) Volume or title not owned . If this reason should be 
proportionately high, this does not necessarily mean 
a library has an inadequate collection. Just by in- 
spection, it can be easily learned if the reason for 

a high figure Is that one or a small group of libraries 
are requesting items that are out-of-scope. Since the 
cost of processing an unfilled request is the same as 
filling one, such inappropriate requests should be 
stopped. If, on the other hand, the items requested 
are in scope, a simple listing of title and publication 
dates can be used to judge whether a major weakness 
exists in resource collections. 

b) Items missing . Although no library administrator likes 
to admit that his housekeeping is poor, a simple count 
of the requests unfilled for this reason can reveal an 
unsuspected failure in any one of several routines that 
are necessary to keep a library operating efficiently; a 
lag in shelving, delays in binding procedures, etc. 
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c) Reference unidentifiable . As already pointed out, the 
processing of an unfillec request is as costly as a 
filled one. Inspection alone can identify if these 
requests are generated from a few libraries, and if so, 
steps can be taken to inform them that they should 
improve their procedures. 

d) I n c i rculat ion . If a proportion of requests are unfilled 
for this reason, it does not take either a great deal of 
analysis to determine (I) if the system of recall of 
materials is not operating adequately, or (ii) if the 
items are heavily used materials which ought to be dupli- 
cated . 

e) Other reasons . The other common causes for non-supply 
of items are usually beyond the control of most admini- 
strative procedures, the items requested are in bindery, 
on reserve or reference, or not yet received. The bindery 
problem has been a knotty one since the advent of library ■ 
binding. The only solution is one of staggering binding 
between library units. Such methods have been utilized 

by large public library systems but have rarely been tried, 
or if tried, maintained, by independent library units. 

s 

A careful tabulation of the non-supply of requests with even just 
a cursory analysis can produce information that is of administrative 
significance. / 

Opt imum del i very serv i ce 

Most library administrators operate with the assumption that 
if they receive no complaints, their library is working efficiently and 
satisfactorily. Non-complaint is probably the most dubious of measure- 
ments available to librarians. It is astonishing the adjustments users 
will make to an Inefficiently operated library or to a library environ- 
ment that appears to be beyond the ability of a librarian to maintain 
control because of external factors. If an RML network is established, 
users wi 1 1 have no alternate 1 ibrary resources . Obviously, each RML must 
operate optimumly. But what standard can be applied to determine what 
is optimum? If a "level of performance" is established, what can be done 
to improve performance? There are but three ways of monitoring a docu- 
ment delivery service that will meet the requirements for measurements 
discussed above; (i) the percent of filled requests of the requests 
processed, (ii) an analysis of unfilled requests, and (iii) measurement 
of timeliness of processing. 

1 . Ratio of filled requests to unf i 1 led requests . Several 
individuals have traced the arbitrary standard of a "90% library" to 
a Public Health Service document. From any studies so far published 
no resource health science library has ever been able to attain this 
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standard. (6) The main reason this is an unrealistic standard is that 
it fails to take into consideration the requests received which are 
beyond the control of the resource library to fill. If a resource li- 
brary has its housekeeping and routine procedures in order which can be 
monitored by analyzing unfilled requests discussed above, and the un- 
filled requests which could not be filled by any library were eliminated 
from the total count, the 90% performance level is probably already 
attained, if not surpassed, by most resource libraries. If the 90% 
level is not attainable because of lack of resources, a relatively modest 
one year's acquisitions program could probably bring up the level to 90% 
performance. A modest acquisitions program is one in which the budget 
would be increased the first year by an amount equivalent to the cost of 
a library's document delivery service and then maintained at one-fourth 
that level thereafter. (This latter statement is intuitively arrived at 
and hence, at this point in time, unchallengeable.) Whether the mystical 
90% performance standard is an adequate one certainly can be easily 
determined by comparing performance among RMLs. However, it must be 
emphasized that such a standard can be realistic only if a more precise 
definition of what should be counted as a legitimate request is made. 

2. T imel i ness . Information, when it is recognized that it is 
needed, is almost always wanted by the user immediately. Instantaneous 
access to documents through any institutional means is impossible today 
and will remain so, except for specialized experimental situations, for 
many years. Obviously, library users have to make compromises which are 
related to the efficiency of libraries to deliver documents. Libraries, 
in turn, should attempt to organize their service to match the work 
patterns of users. Our civilization devises accouterments to deny man's 
diurnal biology, but individuals cannot ignore their biology. Document 
delivery services can be organized along compromised time sequences that 
relate to work patterns. (7) 

Immediate delivery . By immediate is meant within a matter 
of minutes or at most several hours. Such a service can 
only be supplied if a user's library owns the needed docu- 
ment and has it on the shelf. There have been several 
attempts in the past few years to use telefacsimile equip- 
ment to transmit documents between libraries, but these 
have proven to be technically and economically unfeasible. (8) 
It cannot be considered a service goal of an RML system for 
some years. 



(6) For a review and analysis of these studies see. Wood, D.N. and Bower, 
C.A. Survey of Medical Literature Borrowed.... Bui let i n of the 
Medical Library Association 57:^7“83, Jan. 1969- 

(7) For a fuller discussion of this concept see,0rr, R.H., and others. 
Measuring a Library's Capability for Providing Documents. Bulletin 
of the Med i ca 1 Library Assoc iat i on 56:241 267 > July 1 968 ; P i ngs , V . 
Library Service for Health Care. Journal of the Oklahoma State 
Medical Association 6l:l47-153t ApTT 1968 . 

(8) For a report on the latest attempt see, Josey, E.J. The College Library 
in New York 3R System. College and Research Libraries . 30:32-38, 1969. 
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b) Twentv/-four hour del i very . If a user cannot get a document 
within a matter of two or three hours, he will say, "never 
mind", or "can you have it for me tomorrow at this time". 

A 24 hour document delivery service is possible in certain 
metropolitan areas if messenger service is available and 
with the use of TWX in some geographic areas. (9) The only 
alternative to messenger service and tel efacs 1ml 1 e is the 
U.S. mail. The U.S. mall service has not been able to 
deliver mail even in metropolitan areas on an overnight 
basis dependably for many years, much less across state 
lines. Perhaps biomedical institutions can afford the cost 
of TWX, but is not yet a common device. Further, to obtain 
a 24 hour service in an RML system, this would mean all 
requests would have to be processed in a matter of hours. 

c) Forty - eight hour service . Assuming an overnight mall 
delivery service, an 1 nter i nst i tut ional document delivery 
service is possible if the lending library is able to 
process al 1 requests the same day it receives them. 
Considering the quality of clerical help available to li- 
braries in the past year, this ideal performance rate 
appears Impractical. This may, however, be a temporary 
problem because the labor market may ease or the dispensers 
of money may recognize that higher salaries must be paid to 
secure adequate help. Nevertheless, no active library can 
process al 1 requests the same day it receives them. There 
are too many unpredictable aspects in library operation--a 
wanted document is temporarily unavailable because it is 
undergoing some technical processing, is being used by some- 
one in the library, is in the process of being reshelved, etc. 
Surely it is possible nevertheless to set some performance 
rate, say 70%, which should be completely processed the day 
received. Although such a standard could be arbitrarily set 
it would appear more practical to examine the performance 
rate of existing RMLs and from these rates establish a 
minimum level which could be raised as experience and know- 
ledge of the factors involved in document delivery admini- 
stration are codified and standardized. 

d) Seventy-two hours or more . If a user is unable to secure 
a document in less than three days, would it serve his 
purposes just as well if he received it in several days? 

A great deal of effort may be expended to create an organi- 
zation to provide a four day service when in fact, to the 
user, a seven day service would be just as satisfactory. A 
method to collect information which might arrive at least at a 
tentative answer to the question is discussed below. 



(9) The use of telephone to transmit requests is patently Impractical. 
Close to 4200 biomedical Interlibrary loan requests are generated 
each day. 
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3. Availability information . One of the intangible psycho- 
logical factors of document delivery service is that the user wants 
to know what has happened to his request if he does not receive the 
document as he expects. Certainly a courtesy that should be extended 
to all requesters is a prompt report of what actions have been taken; 
it is just as important to report on an inability to supply a document 
as it is to supply it. Such a report will allow a requester to search 
for the information he wants from other sources, or from other documents. 

In measuring the timeliness of a document service, the time required in 
producing a negative response is also of value in measuring performance. 

METHODS ^ TIME MEASUREMENTS 

Two general purposes underly this paper: (i) some measure of 
user cost or user satisfaction must be devised, and (ii) some measure 
of administrative and organizational efficiency must be developed other 
than the mere counting of work done. 

Earlier it was suggested that monitoring of performance could be 
done by merely counting and abstracting information from the request docu- 
ments generated. Some of the data has to be tabulated for the entire 
document delivery processes, other data may be acquired through sampling. 
The measurement of performance against time will require that staff be 
devoted to collecting data. Obviously, this can be done only on a sampl- 
ing basis. To collect such information on a routine basis would not 
only be expensive, but the cost of tabulating and analyzing it would be 
beyond the practical constraints of an RML operation. 

User cost and user sat i sfact ion 

The organizational efficiency of document delivery service has to 
be related to the user market. The user judges the service useful in 
terms of the time he makes his request to the time he actually receives 
it. He is unconcerned with the mechanisms needed to obtain a wanted 
document. The only elements within the control of RMLs are the process- 
ing procedures once the request is received. These procedures can be 
altered to improve delivery time if it is known that certain delays or 
difficulties exist. Further, as a public supported program, some means 
of monitoring the service from the user viewpoint should be developed 
so that it can be demonstrated that the source of delays can be identified. 
If the request is an interlibrary loan, delays may occur at the requesting 
library and not with the RML service. To measure the adequacy of a 
service, therefore, some communication link must be established with the 
user. If the following information were known, an evaluation of user 
satisfaction can be determined as well as provide information for admini- 
strative and management decisions. 

1) Date request actually sent (from postmark) 

2) Date and time request is received 

3) Date and time request dispatched 

4) Date and time request received at requesting library 
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5) Date and time requester actually received document or report 

If time is considered to be a relevant aspect of a document de- 
livery service, it is the usei 's time that is the important time. The 
above five dates and time can show where performance lags exist. 

1) Interlibrary loan requests involve two institutions; 
complaints of slowness of receipt of a document by users 
are not always the fault of the resource library filling 
the request nor the U.S. post office. 

a) Because a requesting library prepares a request on a 
specific day does not always mean it is actually mailed 
that day; internal problems in mail pick-up and de- 
livery within an institution can make the service appear inef- 
ficient at both the requesting and lending institution; 
efficient internal operation of a document delivery service 
may be negated by an inefficient mail pick-up service. 

b) Several alterations in organization can be effected if 
actual time relations are known for different environments, 
for example, (i) TWX can be used to send requests, (ii) ad- 
ditional mail deliveries to the post office can be arranged. 

2) Depending upon sample size, information could be obtained not 
only of a RMLs performance, but that of other institutions as well. 

Data col 1 ect i ng . A method that has been tested to collect data as suggested 
above is to generate an additional record as a request is processed. (10) 

To obtain information from the user, a pre-addressed stamped post card can 
be used which is attached to a sample of requests. Figure 1 is a suggested 
format. The resultant data is then easily manipulated by simple tab equip- 
ment, although if it is determined to be a good monitoring device, it would 
be more practical to make the data amenable for computer manipulation. 

Process i ng time of i nter 1 ibrary and other document requests . A document 
delivery service supported by a RML, as already discussed, is not all within 
its control. A RML, however, should have sufficient control over its 
internal operation to be able to demonstrate and to announce to its users 
(individual and institutional) that of the requests it receives, a certain 
percentage will be processed and sent within a given time. For example, 
knowing the usual transit time within a particular area, a RML should be 
able to tell its users that, say, 80% of the desired documents will be 
received within 48 hours after the request is received at the RML. To make 
such a statement the RML must have control of its processing procedures. 

Data collecting and evaluation . A method has been devised to collect infor- 
mation on the processing of requests that has been tested in six resource 
1 ibrar ies. (1 1) A form accompanies each request as it is processed (see 
Figure 2). The time of certain critical operations is entered on this 

(10) Cruzat, G. Length of Time in Processing Interlibrary Loans. Wayne 
State University Medical Library, Report No. 8, April 1985. 

(11) From unpublished reports prepared for USPHS Contract PH 43-66-540. 
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Original 

Request 



Referral 



Figure 1 

Postmark Time Rec'd Time Dispatch Time 

Date Date Date 

T i me T i me T I me 

Date Date Date 



Dear (name) : 

The Regional Medical Library is attempting to determine the 
adequacy of its organization in its document delivery program. Would 
you please indicate the day and the time you actually received the 
document you requested below and return this preaddressed stamped 
post card to us. 

Date document received Time (a.m. or p.m.) 



< Figure 2 
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form by the processors. After the data has been collected on a repre- 
sentative sample, a measure of the total processing time can be cal- 
culated. Although the median time — the time within which 50% of all 
completed requests are processed- — can serve as a measure of "typical" 
processing time, the data can also be manipulated to show levels of 
performance, say, the time required to process 75% of the requests 
received. The data collected by this method allows one to assess 
request processing and loan processing time separately, and also to 
analyze how the former time intervals differ for requests received via 
different channels (e.g. mail, telephone, TWX) and how the latter inter- 
vals differ for loans furnished as originals and facsimiles. (12) 

Recording processing time data for every request takes effort 
on the part of library staff. However, fairly precise estimates of 
processing time can be obtained routinely by recording data on only a 
sample. Nonstat i st ical considerations dictate a basic plan in which the 
sampling unit is a day's requests; and data recorded on all requests 
received on a random selection of days throughout the year. How many 
days' requests are required will depend on the precision desired, on how 
much the library's load varies, and on how many requests it processes. 

Further considerations . The determination of processing time with the 
objective of establishing a performance standard must.be arrived at 
through better definitions than what are now commonly accepted. This 
relates directly to what has been previously discussed in which it was 
pointed out that all requests involve two parties, the requester and the 
lender. If the requester submits an Incomplete or incorrect request, how 
much responsibility should be taken by the lending library to make up for 
this deficiency? Should all of this work of verification be considered 
as part of the document delivery service or should the time be charged to 
another 1 Ibrary service, for example, reference service? The amount of 
time Involved can indeed alter considerably the performance levels. Say 
a RML, on the average, received 10 such requests per day each of which 
requires three hours of professional time to establish the validity of a 
citation. If the objective Is a same day processing service for al 1 
requests, the resultant organization would require that 10 librarians be 
assigned to verification work each day since each request would have to 
go through a sequence of processing during the day before it is discovered 
that further work is to be done. There is no medical library in the nation 
that has a staff of librarians so large that such a service could be 
organized, consequently this work has to be spread over several days for 
it to be done at all. 

Uncontrollable factors need also to be considered in establishing 
a performance standard. For example, most resource libraries find that 
about 10% of the unfilled requests are due to the fact that the item 
requested is being bound. What time Interval should be allowed in setting 
up a policy with respect to the length of time a request should be held 
waiting for the Item to be returned, and should this waiting time be In- 
cluded in its document delivery performance standard? 



(12) Data-record i ng forms are designed to facilitate keypunching and com- 
puter programs for analyzing the data have been written. 
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SUMMARY AND SUGGESTIONS 



The RML programs are a new institutionalization of library 
service that Is being created through the reorganization of existing 
medical resource libraries which in turn will require alterations in 
procedures in nearly all other medical libraries. The service receiv- 
ing first priority is that of document delivery. Since this is a 
federally sponsored program, monitoring devices must be established 
to determine if the program is in fact accomplishing the alrr, of improv- 
ing access to medical information, and if not, what constraints now 
operating in maintaining medical libraries can be altered to accomplish 
the aim. 



Two general perspectives are used on which to base a design for 
measurement of document delivery services and on which to base performance 
standards, (1) what is the cost to the user in using this service, and 
(ii) how efficient are the procedures in providing the service. One 
overriding factor governs the design of measuring this RML service; a 
more precise definition of what variables are to be included with differ- 
ent measurements is necessary before any realistic performance standards 
can be established. A document delivery service Involves two parties, 
the requester and the lender. Without distinguishing responsibilities 
between the two parties, discrepancies in measurements result. The data 
then become less useful for management and planning purposes. Further, 
any measurements devised must remain within practical limitations, and 
except for unusual studies, be established as a routine operation. 

Three sources of data are suggested: 

1) The records generated in making requests, 

2) Two special means of data collecting requiring separate 
routines for collecting: 

a) The one determines the user time involved, 

b) The other monitors the operational procedures in 
providing the service. 

All three sources can provide information on user cost and on admini- 
strative functions. Assuming a better definition of document delivery 
service than is now generally accepted is agreed upon the following 
priorities of analyzing data are suggested. 



1) Determine who uses the service; this is needed for two 
reasons, (i) to justify the continuation of the service, 
and (il) to determine relative emphasis of Investment in 
providing supportive functions. 






2) Analyze the reasons requests are not filled which can 
serve as (j) a monitoring device for service given and 
to establish better criteria for defining the service, 
and (ii) a means for determining the adequacy of resources 
and establishing performance standards. 

3) Collect and analyze data on the request processing and 
loan processing time which can serve as a means of monitor- 
ing performance and to provide a basis for planning improve- 
ments through administrative and organizational means, 

4) Establish the actual user real time cost In using the 
service for different environments. 

Many other aspects of document delivery service can be studied 
and analyzed. Although they should not be considered as unlmportaf.t , 
these aspects can be studied more efficiently when the results are ob- 
tained from the four priority areas delineated. 



